import AuthorDetails from '@site/src/components/AuthorDetails';

## Togetherß

#### Adapted from documentation provided by https://github.com/insop

### Prerequisites

- Together `api_key` and `api_base` (_**for non-cached examples**_).
Set these within your developer environment `.env` as follows:

```
TOGETHER_API_BASE = ...
TOGETHER_API_KEY = ...
```

which will be retrieved within the Together Client as:
```python
self.api_base = os.getenv("TOGETHER_API_BASE")
self.token = os.getenv("TOGETHER_API_KEY")
```

### Setting up the Together Client

The constructor initializes the `HFModel` base class to support the handling of prompting models. This requires the following parameters:

**Parameters:**
- `model` (_str_): ID of model hosted on Together endpoint.
- `**kwargs`: Additional keyword arguments to configure the Together client.

Example of the Together constructor:

```python
class Together(HFModel):
    def __init__(self, model, **kwargs):
```

### Under the Hood

#### `_generate(self, prompt, use_chat_api=False, **kwargs):`

**Parameters:**
- `prompt` (_str_): Prompt to send to Together.
- `use_chat_api` (_bool_): Flag to use the Together Chat models endpoint. Defaults to False.
- `**kwargs`: Additional keyword arguments for completion request.

**Returns:**
- `dict`: dictionary with `prompt` and list of response `choices`.

Internally, the method handles the specifics of preparing the request prompt and corresponding payload to obtain the response. 

The Together token is set within the request headers to ensure authorization to send requests to the endpoint. 

If `use_chat_api` is set, the method sets up Together url chat endpoint and prompt template for chat models. It then retrieves the generated JSON response and sets up the `completions` list by retrieving the response's `message` : `content`. 

If `use_chat_api` is not set, the method uses the default Together url endpoint. It similarly retrieves the generated JSON response and but sets up the `completions` list by retrieving the response's `text` as the completion.

Finally, after processing the requests and responses, the method constructs the response dictionary with two keys: the original request `prompt` and `choices`, a list of dictionaries representing generated `completions` with the key `text` holding the response's generated text.

### Using the Together client

```python
together = dspy.Together(model="mistralai/Mistral-7B-v0.1")
```

### Sending Requests via Together Client

1) _**Recommended**_ Configure default LM using `dspy.configure`.

This allows you to define programs in DSPy and simply call modules on your input fields, having DSPy internally call the prompt on the configured LM.

```python
dspy.configure(lm=together)

#Example DSPy CoT QA program
qa = dspy.ChainOfThought('question -> answer')

response = qa(question="What is the capital of Paris?") #Prompted to together
print(response.answer)
```

2) Generate responses using the client directly.

```python
response = together(prompt='What is the capital of Paris?')
print(response)
```

***

<AuthorDetails name="Arnav Singhvi"/>